Distribution channel using audio/visual recognition

ABSTRACT

Systems and methods are provided for providing a platform to provide virtual storefronts to consumers. Environmental elements are associated with specific consumer services on computer server. A user in the environment takes audio or visual recordings of an environmental element and uploads the recordings to the server. The server determines the appropriate consumer service associated with the recorded environmental element and provides the user with a reference to the service. Accordingly, any space, such as a bus stop, subway or train terminal, movie theater, or airport, may be turned into a virtual store.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.15/833,936, filed Dec. 6, 2017, which is a continuation of U.S. patentapplication Ser. No. 13/844,414, filed on Mar. 15, 2013, which claimsthe benefit of U.S. Provisional Application No. 61/739,664, filed Dec.19, 2012.

Technical Field

The present disclosure relates generally to marketing systems, and moreparticularly, some embodiments related to systems and methods forproviding product distribution channels.

Description of Related Art

Millions of dollars are spent on marketing every year. Ever greaterefficiency to deliver immediate commercial opportunity for consumers isdesired. Additionally, analytics/data of interaction and usage is animportant tool for marketing, and methods for gathering such data aredesired.

Two dimensional barcodes such as QR codes are used in consumeradvertising. QR codes can encode information such as uniform resourcelocators (URLs) leading to marketing-related websites or internetprovided services. Users can install an app on their phones or othermobile devices allowing the QR codes to be read by the mobile device.The users may then be directed to the marketing-related websites orinternet provided services at the URLs encoded by the QR codes.

BRIEF SUMMARY

A platform to provide virtual storefronts to consumers is disclosed.Environmental elements are associated with specific consumer services ona computer server. A user in the environment takes audio or visualrecordings of an environmental element and uploads the recordings to theserver. The server determines the appropriate consumer serviceassociated with the recorded environmental element and provides the userwith a reference to the service. Accordingly, any space, such as a busstop, subway or train terminal, movie theater, or airport, may become avirtual storefront.

Other features and aspects of the disclosure will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings, which illustrate, by way of example, the featuresin accordance with various implementations.

BRIEF DESCRIPTION OF THE FIGURES

The figures are provided for purposes of illustration only and merelydepict typical or example embodiments. They do not limit the breadth,scope, or applicability of the invention.

FIG. 1 illustrates an example system for providing a distributionchannel.

FIG. 2 illustrates an example method of providing a distribution channelusing the system of FIG. 1.

FIG. 3 illustrates an example implementation of the system of FIG. 1.

FIG. 4 illustrates an example deployment of a virtual storefrontplatform using the system of FIG. 1.

FIG. 5 illustrates a second example deployment of a virtual storefrontplatform using the system of FIG. 1.

FIG. 6 illustrates a third example deployment of a virtual storefrontplatform using the system of FIG. 1.

FIG. 7 illustrates an example computing module that may be used toimplement various features of the system and methods disclosed herein.

DETAILED DESCRIPTION OF THE EMBODIMENTS OF THE DISCLOSURE

The disclosure provides a new platform by which to engage consumers,thereby extending the consumer experience and offering differentiationusing existing marketing elements and physical product. In someimplementations, image recognition technology is used to enhance theviewer's experience with a product offering. For example, adding imagerecognition technology to marketing elements (bus shelters, billboards,standees and ads (print and TV)) and physical product adds a level ofinteractivity, education, and commercialization, which would not bepossible otherwise.

Using visual/image recognition technology combined with elements ofaugmented reality technology, a new distribution channel can be createdby taking advantage of existing marketing materials (e.g. bus shelters,standees, billboards) and physical product (BDs and DVDs) in themarketplace. A new platform for consumers and movie-goers is created toengage with is thereby created. This yields a new commercial path andrevenue generation, for example but not limited to, purchasing movietickets from a bus shelter or digital products from a DVD, BD or VHS.

This platform may allow creation of marketing elements such as a“virtual wall” selling merchandize via image recognition technology inareas of tight real estate or places product wouldn't normally be sold.

Utilizing bus shelters, movie posters, and other marketing elements canhelp put more people in theater seats. The Virtual Wall can help enticemovie-goers once in the theater and introduce digital commercializationto theaters in a very cost effective manner. Examples include movietheaters (TDS product and/or DVDs & BDs) and stores (DVDs and BDs).

In one implementation, the consumer downloads a visual recognition appto their device (e.g., a smartphone or tablet) and interacts withproducts using image recognition technology. For example, the productmay comprise a movie poster, product advertisement, bus shelter, productlabel, product packaging, or any other element in the environment. Insome implementations, products that are compatible with the applicationhave a logo or other information element on the product. However, thelogo or other information element does not have to be present. Rather,the logo simply signals that interactivity is possible. Once the triggerimage is recognized an interface appears on the device with the variouspurchase options. The interface can be scaled and modified based oncreative or strategy. All platforms available such as, but not limitedto iOS® or Android®. For example, the interface can be provided througha web browser on the device.

Examples of interactivity with a virtual wall or marketing elementinclude: ordering movie tickets from a movie poster, bus shelter or TVad; disc to digital up-grades from movie posters or physical product;registering or claiming rewards on reward programs such as a movierewards program; pre-ordering or purchasing a Blu-Ray/DVD from movieposters or existing physical product; finding out more about a movie orservice, such as watching trailers (informational data that may beeither programmed or sourced from existing web-based services). Anyfunctionality described herein may also be available through visualrecognition of audio visual content on tv/video (utilizing audiorecognition and or visual recognition footprints). Additional servicesinclude augmented reality. For example, a user may take a picture with avirtual movie character using Facebook®, Email or Twitter® to share andpotentially entice others to engage and consume

Examples of marketing materials used (full marketing mix) include: movieposters or bus shelters; physical product-DVD, BD and VHS; audio via TVspot/Trailer; standees; and billboards.

Augmented reality and image recognition technology has primarily beenused in marketing and brand awareness driven initiatives. However, thesetechnologies have been unable to leverage existing marketing elementsfor such initiatives. For example, QR codes are obtrusive and must bedeveloped specifically for a given initiative. Additionally, thesetechnologies have not been utilized for distribution.

FIG. 1 illustrates a system for providing a consumer engagementplatform. The system comprises a server 101 having a data storage 102.The server 101 may comprise any product providing resources to clientmachines. For example, the server 101 may comprise dedicated computers,workstations, cloud servers, server clusters, or other serverimplementations. The data storage 102 may comprise local or networkstorage. The server 101 stores data associated with one or moreenvironmental elements 107 in data storage 102. Additionally, the server101 stores associations between the environmental elements 107 andspecific services provided by provider 108. In some embodiments, theserver 101 also associates location data with environmental elements 107and the services provided by provider 108.

The server 101 is connected via a network 103, such as the internet, toa user device 106. The user device 106 may comprise various computingdevices, such as desktop or laptop computers, tablets, mobile phones,media players, or other network connected devices. The user device 106comprises an input 104 that is capable of recording 105 environmentalelements 107. For example, the input 104 may comprise a microphone ordigital camera. The input 104 may be configured to record anenvironmental element 107 upon manual command from the user of device106 (for example, taking a picture) or may be configured to passivelyrecord the environmental element 107 (for example, always-on microphonefunctionality). In some implementations, the user device 106 maycomprise a GPS 109 or other module capable of providing locationinformation.

A service provider 108 is also connected to network 103. The serviceprovider 108 may comprise a server, host, merchant, or othernetwork-connected service provider. For example, the service provider108 may provide a digital storefront for users of user device 106 topurchase products such as merchandise(such as clothing, or toys)physical copies of media (such as films, books, or songs), digitalcopies of media, digital streams of media, tickets for events orentertainment venues, preorders of upcoming releases. Such products maybe purchased for the user of device 108 or may be gifted to others. Theservice provider 108 may be equipped to allow the purchase of productsusing a variety of currencies, such as credit cards, virtual currencies,social networking points, simple message service (SMS) text paymentmethods, PayPal®, or carrier bills. As further examples, the serviceprovider 108 may provide other services such as club membership, socialnetworking services, movie times, streaming audiovisual content,information retrieved from other existing web-based services, augmentedreality services, or other services to engage users. In someimplementations, the server 101 and the service provider 108 may be thesame entity or may be under common control. In further implementations,the service provider 108 may provide a service to the user device 106based on location data received from the user device 106. For example,the service provider 108 might provide an option to purchase a movieticket if location data indicates the user device 106 is in the vicinityof a movie theater.

Environmental element 107 may comprise any audio or visual presentationthat may be encountered by the user of device 106. For example, theenvironmental element 107 may comprise a non-encoded design element,such as a graphic, a marketing element, a bus shelter, a standee, anadvertisement, a physical product, a movie poster, a toy, an audio orvisual clip from a movie or television show, a commercial, or a packagesuch as a DVD/Blu-Ray or VHS case. In some implementations, theenvironmental element 107 further comprises an icon, graphic, or otherindication that there are services associated with the environmentalelement 107. The environmental element 107 may comprise pre-existingmaterials or may comprise materials developed to be associated with theservices provided by service provider 108.

FIG. 2 illustrates a method of operation of a user device 106 and aserver 101. In step 201, the server 101 stores service-associatedinformation in data storage 102. The service-associated information isinformation sufficient to allow the server 101 to perform recognitionprocesses on information provided by user devices. For example, theservice-associated information may comprise copies of the environmentalelements 107, images of the environmental elements 107, recordings ofenvironmental elements 107, or recognition usable primitives derivedfrom the environmental elements 107. The services associated with theservice-associated information can be any services provided by provider108.

In step 202, the user device 106 downloads a mobile application (app) toallow the user device 106 to interface with the server 101. In variousembodiments, the app can be downloaded from server 101, service provider108, or a third party hosted app store. In other embodiments, step 202can be omitted and the user device 106 can interact with the server 101using other methods, such as through a web browser.

In step 203, the user device 106 records an environmental element in thevicinity of the user device 106. For example, the user device 106 maytake a picture of a graphic, such as a movie poster, bus shelter, orkiosk. As another example, the user device 106 may take an audiorecording of a song or commercial. As another example, the user device106 may take a video recording of a portion of a movie, commercial, ortelevision show. In some implementations, the user device 106 performsstep 203 in response to a command from the user of device 106. In otherimplementations, the user device 106 performs step 203 automatically.

In step 204, the user device 106 uploads information obtained from thestep 203 of recording the environmental element. For example, the userdevice 106 may upload the data obtained by the step 203 of recording. Asanother example, the user device 106 may perform some preprocessing aspart of step 203. The preprocessing may reduce the amount of data sentto server 101 or may assist the server 101 in performing the audio orvisual recognition.

In step 205, the server 101 receives the user-obtained informationuploaded in step 204. In step 206, the server 101 performs an audio orvisual recognition process on the information obtained in step 205. Theaudio or visual recognition process identifies the environmental elementrecorded in step 203. Additionally, during step 205, the server 101identifies the service or services associated with the identifiedenvironmental element.

In step 207, the server 101 provides a reference to the identifiedservice to the user device 106. For example, the reference may be a URLto the service. In step 208, the user device 106 receives the reference.In step 209, the user device interfaces with the service provided byservice provider 108. In other implementations, steps 207 and 208 may beomitted, and the server 101 may provide the identified service directlyto the user device 106.

FIG. 3 illustrates a system and method for providing a distributionchannel using image recognition and augmented reality technology. Here,image recognition is added to existing marketing elements (bus shelters,billboards, standees and ads (print and TV)) and physical product adds alevel interactivity, education, and commercialization, which would notbe possible otherwise.

In this implementation, one or more images 301 are uploaded and storedin a cloud network 302 or other server. For example, with reference toFIG. 1, images 301 may be an implementation of information related to anenvironmental element 107 and cloud network 302 may be an implementationof server 101 and data storage 102. Services are associated with theimages 301 and the associated services are stored in the server 302. Forexample, a separate service may be stored for each image. Alternatively,combinations of images 301 may have associated services. For example, aparticular array of images 301 may be recognized by server 302 ascorresponding to a particular digital storefront provided by particularprovider 108.

During operation, a user uses a device 304 to take an image or recordingof a product 303 displaying one or more of the images 301. For example,this step may be an implementation of a user device 106 recording 105 anenvironmental element 107. The device 304 transmits the recordedinformation to the server 302. The server 302 then recognizes the images301 on product 303 and transmits a direction command to the device 304.For example, the direction command may be an implementation of areference to a service provided by provider 108.

The device 304 Is then directed to the associated service, such as amobile storefront. For example, the product 303 may comprise a displayof a set of movie posters or other images 301. The device 304 takes apicture of the set of movie images 301, and the server 302 may provide astorefront allowing the user to preorder any one of the movies.Alternatively, the sever 302 may direct the device 304 to a websiteoffering such services. The electronic stores provided by the server 302or other website may use a variety of currencies, such as virtualcurrency, Facebook® points, SMS payment, PayPal®, carrier bills, ormovie reward program points. As another example, the server 302 may senda list of recognized movies, including links to product purchase sites,movie trailers, or other services. For example, the set of images 301may be virtual store.

Accordingly, the system creates a new platform by which to engageconsumers thereby extending the consumer experience and offeringdifferentiation using existing marketing elements and physical product.

FIG. 2 illustrates an example deployment of a distribution channel usinga movie poster 403 at a bus stop 407. This deployment may be implementedusing the system of FIG. 1. The movie poster 403 may comprise anenvironmental element 107. An image used for a movie poster 403 isstored in the server 101 on data storage 102 and associated with variousservices 401, 402, 404, 405, 406 provided by one or more providers 108.For example, a movie ticket purchase service 402, a movie review website401, a pre-order 406 of the movie, a service for claiming rewards pointsor purchasing merchandise (e.g., clothes or toys) 404, or social networkfeatures 405. Accordingly, when a user views the poster 403, the usermay use a user device to take a picture of the poster 403 and upload thepicture to the server 101. The server 101 then recognizes the poster 403and directs the user to one of the services 401, 402, 404, 405, or 406.

FIG. 5 illustrates an example deployment in a theater. A poster, image,other marketing element, or standee 504 is provided in a theater. Thisdeployment may also be implemented using the system of FIG. 1. Theposter or standee 504 image is uploaded onto a cloud or other server 507and associated with services 501, 502, 503, 505, 506, 508. The poster504 may also be equipped with a logo or decal 509. The logo or decal 509may indicate to users that the poster 504 is associated with services501, 502, 503, 505, 506, 508, and can be used with their user devices106 or applications stored on their user devices 106.

When the user takes a picture of the marketing element 504, the pictureis uploaded to the server 507. At the server 507, the server 507recognizes the image 504 and directs the device to one of the services502, 503, 505, 506, 508. As examples, the services may include: movieticket purchase services 503; movie or product pages 501, includingtrailers 502; social networking sites 506, for example, the service maybe a link to share the product on Facebook® or Twitter®; movie rewardssites 508; or electronic stores 505.

FIG. 6 illustrates another example deployment that may be implementedusing the system of FIG. 1. Here, a virtual wall 601 is deployed in anenvironment 602 such as a theater, airport, or hotel. The wall virtual601 presents a variety of content related images 603, such as images ofcover art. Each image 603 is associated with a corresponding service,for example, purchase of an electronic copy of the content correspondingto the image 603. A user may then use the virtual wall 601 to selectwhich product the user wishes to purchase. The user may then take apicture of that product and be sent an electronic copy of the content.As another example, the user may take a picture of the entire virtualwall 601, or a portion thereof, and be presented with service optionscorresponding to all of the images 603 of the entire virtual wall 601 orportion thereof.

As used herein, the term module might describe a given unit offunctionality that can be performed in accordance with one or moreembodiments of the present application. As used herein, a module mightbe implemented utilizing any form of hardware, software, or acombination thereof. For example, one or more processors, controllers,ASICs, PLAs, PALs, CPLDs, FPGAs, logical components, software routinesor other mechanisms might be implemented to make up a module. Inimplementation, the various modules described herein might beimplemented as discrete modules or the functions and features describedcan be shared in part or in total among one or more modules. In otherwords, as would be apparent to one of ordinary skill in the art afterreading this description, the various features and functionalitydescribed herein may be implemented in any given application and can beimplemented in one or more separate or shared modules in variouscombinations and permutations. Even though various features or elementsof functionality may be individually described or claimed as separatemodules, one of ordinary skill in the art will understand that thesefeatures and functionality can be shared among one or more commonsoftware and hardware elements, and such description shall not requireor imply that separate hardware or software components are used toimplement such features or functionality.

Where components or modules of the application are implemented in wholeor in part using software, in one embodiment, these software elementscan be implemented to operate with a computing or processing modulecapable of carrying out the functionality described with respectthereto. One such example computing module is shown in FIG. 7. Variousembodiments are described in terms of this example-computing module 700.After reading this description, it will become apparent to a personskilled in the relevant art how to implement the application using othercomputing modules or architectures.

Referring now to FIG. 7, computing module 700 may represent, forexample, computing or processing capabilities found within desktop,laptop and notebook computers; hand-held computing devices (PDA's, smartphones, cell phones, palmtops, etc.); mainframes, supercomputers,workstations or servers; or any other type of special-purpose orgeneral-purpose computing devices as may be desirable or appropriate fora given application or environment. Computing module 700 might alsorepresent computing capabilities embedded within or otherwise availableto a given device. For example, a computing module might be found inother electronic devices such as, for example, digital cameras,navigation systems, cellular telephones, portable computing devices,modems, routers, WAPs, terminals and other electronic devices that mightinclude some form of processing capability.

Computing module 700 might include, for example, one or more processors,controllers, control modules, or other processing devices, such as aprocessor 704. Processor 704 might be implemented using ageneral-purpose or special-purpose processing engine such as, forexample, a microprocessor, controller, or other control logic. In theillustrated example, processor 704 is connected to a bus 702, althoughany communication medium can be used to facilitate interaction withother components of computing module 700 or to communicate externally.

Computing module 700 might also include one or more memory modules,simply referred to herein as main memory 708. For example, preferablyrandom access memory (RAM) or other dynamic memory, might be used forstoring information and instructions to be executed by processor 704.Main memory 708 might also be used for storing temporary variables orother intermediate information during execution of instructions to beexecuted by processor 704. Computing module 700 might likewise include aread only memory (“ROM”) or other static storage device coupled to bus702 for storing static information and instructions for processor 704.

The computing module 700 might also include one or more various forms ofinformation storage mechanism 710, which might include, for example, amedia drive 712 and a storage unit interface 720. The media drive 712might include a drive or other mechanism to support fixed or removablestorage media 714. For example, a hard disk drive, a floppy disk drive,a magnetic tape drive, an optical disk drive, a CD or DVD drive (R orRW), or other removable or fixed media drive might be provided.Accordingly, storage media 714 might include, for example, a hard disk,a floppy disk, magnetic tape, cartridge, optical disk, a CD or DVD, orother fixed or removable medium that is read by, written to or accessedby media drive 712. As these examples illustrate, the storage media 714can include a computer usable storage medium having stored thereincomputer software or data.

In alternative embodiments, information storage mechanism 710 mightinclude other similar instrumentalities for allowing computer programsor other instructions or data to be loaded into computing module 700.Such instrumentalities might include, for example, a fixed or removablestorage unit 722 and an interface 720. Examples of such storage units722 and interfaces 720 can include a program cartridge and cartridgeinterface, a removable memory (for example, a flash memory or otherremovable memory module) and memory slot, a PCMCIA slot and card, andother fixed or removable storage units 722 and interfaces 720 that allowsoftware and data to be transferred from the storage unit 722 tocomputing module 700.

Computing module 700 might also include a communications interface 724.Communications interface 724 might be used to allow software and data tobe transferred between computing module 700 and external devices.Examples of communications interface 724 might include a modem orsoftmodem, a network interface (such as an Ethernet, network interfacecard, WiMedia, IEEE 802.XX or other interface), a communications port(such as for example, a USB port, IR port, RS232 port Bluetooth®interface, or other port), or other communications interface. Softwareand data transferred via communications interface 724 might typically becarried on signals, which can be electronic, electromagnetic (whichincludes optical) or other signals capable of being exchanged by a givencommunications interface 724. These signals might be provided tocommunications interface 724 via a channel 728. This channel 728 mightcarry signals and might be implemented using a wired or wirelesscommunication medium. Some examples of a channel might include a phoneline, a cellular link, an RF link, an optical link, a network interface,a local or wide area network, and other wired or wireless communicationschannels.

In this document, the terms “computer program medium” and “computerusable medium” are used to generally refer to transitory ornon-transitory media such as, for example, memory 708, storage unit 720,storage media 714, and channel 728. These and other various forms ofcomputer program media or computer usable media may be involved incarrying one or more sequences of one or more instructions to aprocessing device for execution. Such instructions embodied on themedium, are generally referred to as “computer program code” or a“computer program product” (which may be grouped in the form of computerprograms or other groupings). When executed, such instructions mightenable the computing module 700 to perform features or functions of thepresent application as discussed herein.

Although described above in terms of various exemplary embodiments andimplementations, it should be understood that the various features,aspects and functionality described in one or more of the individualembodiments are not limited in their applicability to the particularembodiment with which they are described, but instead can be applied,alone or in various combinations, to one or more of the otherembodiments of the application, whether or not such embodiments aredescribed and whether or not such features are presented as being a partof a described embodiment. Thus, the breadth and scope of the presentapplication should not be limited by any of the above-describedexemplary embodiments.

Terms and phrases used in this document, and variations thereof, unlessotherwise expressly stated, should be construed as open ended as opposedto limiting. As examples of the foregoing: the term “including” shouldbe read as meaning “including, without limitation” or the like; the term“example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof; the terms “a” or“an” should be read as meaning “at least one,” “one or more” or thelike; and adjectives such as “conventional,” “traditional,” “normal,”“standard,” “known” and terms of similar meaning should not be construedas limiting the item described to a given time period or to an itemavailable as of a given time, but instead should be read to encompassconventional, traditional, normal, or standard technologies that may beavailable or known now or at any time in the future. Likewise, wherethis document refers to technologies that would be apparent or known toone of ordinary skill in the art, such technologies encompass thoseapparent or known to the skilled artisan now or at any time in thefuture.

The presence of broadening words and phrases such as “one or more,” “atleast,” “but not limited to” or other like phrases in some instancesshall not be read to mean that the narrower case is intended or requiredin instances where such broadening phrases may be absent. The use of theterm “module” does not imply that the components or functionalitydescribed or claimed as part of the module are all configured in acommon package. Indeed, any or all of the various components of amodule, whether control logic or other components, can be combined in asingle package or separately maintained and can further be distributedin multiple groupings or packages or across multiple locations.

Additionally, the various embodiments set forth herein are described interms of exemplary block diagrams, flow charts and other illustrations.As will become apparent to one of ordinary skill in the art afterreading this document, the illustrated embodiments and their variousalternatives can be implemented without confinement to the illustratedexamples. For example, block diagrams and their accompanying descriptionshould not be construed as mandating a particular architecture orconfiguration.

1. A system comprising: an environmental element adapted to display aplurality of non-encoded design elements, wherein the plurality ofnon-encoded design elements have been associated with service-associatedvisual information stored on a server; and a user device adapted tocapture the plurality of non-encoded design elements of theenvironmental element and to send captured non-encoded design elementsof the environmental element to the server, wherein the server isadapted to perform visual recognition for the captured non-encodedgraphics and to provide the user device with a service based on acombination of two or more of the captured non-encoded design elements.2. The system of claim 1, wherein each of the non-encoded designelements is associated on the server with a different service.
 3. Thesystem of claim 1, wherein the service comprises an electronic store, asocial network, or a video streaming network.
 4. The system of claim 1,wherein the service is an augmented reality application.
 5. The systemof claim 1, wherein the environmental element is a marketing element. 6.The system of claim 1, wherein the user device is configured to upload alocation of the user device to the server, and wherein the service isassociated with the location of the user device.
 7. A system comprising:an environmental element adapted to display a plurality of non-encodeddesign elements capturable by a user device, wherein the plurality ofnon-encoded design elements have been associated with service-associatedvisual information; and a server adapted to store the service-associatedvisual information, to receive the plurality of non-encoded designelements from the user device, to perform visual recognition for theplurality of non-encoded graphics, and to provide the user device with aservice based on a combination of two or more of the non-encoded designelements.
 8. The system of claim 7, wherein each of the non-encodeddesign elements is associated on the server with a different service. 9.The system of claim 7, wherein the service comprises an electronicstore, a social network, or a video streaming network.
 10. The system ofclaim 7, wherein the service is an augmented reality application. 11.The system of claim 7, wherein the environmental element is a marketingelement.
 12. The system of claim 7, wherein the server receives alocation of the user device from the user device, and wherein theservice is associated with the location of the user device.